Varieties of Meta-cognition in Natural and Artificial Systems

Some AI researchers aim to make useful machines, including robots. Others aim to understand general principles of information-processing machines whether natural or artificial, often with special emphasis on humans and human-like systems: They primarily address scientific and philosophical questions rather than practical goals. However, the tasks required to pursue scientific and engineering goals overlap considerably, since both involve building working systems to test ideas and demonstrate results, and the conceptual frameworks and development tools needed for both overlap. This paper, partly based on requirements analysis in the CoSy robotics project, surveys varieties of meta-cognition and draws attention to some types that appear to play a role in intelligent biological individuals (e.g. humans) and which could also help with practical engineering goals, but seem not to have been noticed by most researchers in the field. There are important implications for architectures and representations. Varieties of Requirements and Designs The workshop manifesto (Cox and Raja 2007) states “The 21st century is experiencing a renewed interest in an old idea within artificial intelligence that goes to the heart of what it means to be both human and intelligent. This idea is that much can be gained by thinking of one’s own thinking. Metareasoning is the process of reasoning about reasoning itself.” This implies that the idea is not restricted to engineering goals, but includes the scientific and philosophical study of humans. As is clear from Cox (2005) the scientific concern with metacognition in AI goes back to the founders. Scientific and philosophical aims have always been my primary reason for interest in AI, including meta-cognitive mechanisms (e.g. in chapters 6 and 10 of my 1978). Study of other animals should also be included, since humans are products of a process that produced organisms with many sizes, shapes, habitats, competences, and social organisations; and we cannot expect to understand all the design tradeoffs in humans unless we compare alternatives. Such comparisons could also be of great importance for biology/ethology. That would involve studying both the space of requirements (niche space) and the space of designs that can be assessed against those requirements (deCopyright c © 2008, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. sign space). Assessment need not be production of a measurement, e.g. a number or a total ‘fitness’ ordering. Instead comparisons could produce structured descriptions of strengths and weaknesses in various conditions and in relation to various functions (like the more useful consumer reports and Minsky (1963)). One way to do that comparative study is to attempt analytically to retrace the steps of biological evolution. Simply simulating evolution does not necessarily yield any new understanding of design-options or tradeoffs, however impressive the end-products. But retrospective analysis does not need to follow the chronology of evolution: working backward analytically may be as informative as working forward, in studying both evolution and individual development. Since the whole evolutionary process was so long, so broad (because many things evolved in parallel) and so intricate, it may be most fruitful to attempt to identify major design discontinuities, producing before-after comparisons of both requirements and designs and analysing their implications, both for science (including psychology, biology, neuroscience) and for engineering. A partial, purely illustrative, survey of this sort was presented in Sloman (2007a). Philosophy, especially conceptual analysis, will inevitably be involved in the process. This paper attempts to identify issues to be addressed in such analytical comparative investigations, starting from a collection of design features of humans that are not widely recognized. Beyond the Manifesto Figure 1: From the workshop manifesto One implication of the generalisation to biological phenomena (and human-like robots) is that the “ground level” referred to in the manifesto (Fig. 1) may include arbitrarily complex physical and social environments. In humans, while awake, there are sensors and effectors continuously Figure 2: Dynamical subsystems vary in many ways including degree of environmental coupling, speed of change, whether continuous or discrete, what is represented, etc. coupled to the environment: i.e. the coupling does not alternate between being on and off while more central processes analyse sensory inputs or decide what to do. Consequently, instead of an “action-perception cycle”, we need an architecture with concurrent processes of many kinds, which can interact with one another. (Even a single-cpu computer can support concurrent enduring processes because, while the cpu is shared, the majority of the state of each process endures in memory. However, some kinds of concurrency may require specialised hardware, e.g. for continuous control.) So the arrows, instead of representing cyclic transitions between states represented by the boxes, as in flow charts, must represent flow of information and control between enduring sub-systems operating at different levels of abstraction, on different time-scales, some changing continuously, others discretely, and performing different functions (as in Figs 2 and 3). This has deep implications for forms of representation, algorithms, and architectures, and for possible interactions and conflicts between sub-systems. Such concurrency was out of the question in the early days of AI: computers were far too slow and had far too little memory. If finding the rim of a teacup in an image takes 20 minutes, a robot cannot perceive and act concurrently. There are also application domains where continuous monitoring and control are out of the question because everything is discrete and all objects are static, e.g. most of the internet.